A study of dependency features of spike trains through copulas

Simultaneous recordings from many neurons hide important information and the connections characterizing the network remain generally undiscovered despite the progresses of statistical and machine learning techniques. Discerning the presence of direct links between neuron from data is still a not completely solved problem. To enlarge the number of tools for detecting the underlying network structure, we propose here the use of copulas, pursuing on a research direction we started in [1]. Here, we adapt their use to distinguish different types of connections on a very simple network. Our proposal consists in choosing suitable random intervals in pairs of spike trains determining the shapes of their copulas. We show that this approach allows to detect different types of dependencies. We illustrate the features of the proposed method on synthetic data from suitably connected networks of two or three formal neurons directly connected or influenced by the surrounding network. We show how a smart choice of pairs of random times together with the use of empirical copulas allows to discern between direct and un-direct interactions

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Among the various architectures of Recurrent Neural Networks, Echo State Networks (ESNs) emerged due to their simplified and inexpensive training procedure. These networks are known to be sensitive to the setting of hyper-parameters, which critically affect their behaviour. Results show that their performance is usually maximized in a narrow region of hyper-parameter space called edge of chaos. Finding such a region requires searching in hyper-parameter space in a sensible way: hyper-parameter configurations marginally outside such a region might yield networks exhibiting fully developed chaos, hence producing unreliable computations. The performance gain due to optimizing hyper-parameters can be studied by considering the memory--nonlinearity trade-off, i.e., the fact that increasing the nonlinear behavior of the network degrades its ability to remember past inputs, and vice-versa. In this paper, we propose a model of ESNs that eliminates critical dependence on hyper-parameters, resulting in networks that provably cannot enter a chaotic regime and, at the same time, denotes nonlinear behaviour in phase space characterised by a large memory of past inputs, comparable to the one of linear networks. Our contribution is supported by experiments corroborating our theoretical findings, showing that the proposed model displays dynamics that are rich-enough to approximate many common nonlinear systems used for benchmarking

A characterization of the Edge of Criticality in Binary Echo State Networks

Echo State Networks (ESNs) are simplified recurrent neural network models composed of a reservoir and a linear, trainable readout layer. The reservoir is tunable by some hyper-parameters that control the network behaviour. ESNs are known to be effective in solving tasks when configured on a region in (hyper-)parameter space called \emph{Edge of Criticality} (EoC), where the system is maximally sensitive to perturbations hence affecting its behaviour. In this paper, we propose binary ESNs, which are architecturally equivalent to standard ESNs but consider binary activation functions and binary recurrent weights. For these networks, we derive a closed-form expression for the EoC in the autonomous case and perform simulations in order to assess their behavior in the case of noisy neurons and in the presence of a signal. We propose a theoretical explanation for the fact that the variance of the input plays a major role in characterizing the EoC

Learn to Synchronize, Synchronize to Learn

In recent years, the machine learning community has seen a continuous growing interest in research aimed at investigating dynamical aspects of both training procedures and perfected recurrent models. Of particular interest among recurrent neural networks we have the Reservoir Computing (RC) paradigm characterized by conceptual simplicity and fast training scheme. Yet, the guiding principles under which RC operates are only partially understood. In this work, we study the properties behind learning dynamical systems and propose a new guiding principle based on Generalized Synchronization (GS), granting to learn a generic task with RC architectures. We show that the well-known Echo State Property (ESP) implies and is implied by GS, so that theoretical results derived from the ESP still hold when GS does. However, by using GS one can profitably study the RC learning procedure by linking the reservoir dynamics with the readout training. Notably, this allows us to shed light on the interplay between the input encoding performed by the reservoir and the output produced by the readout optimized for the task at hand. In addition, we show that - as opposed to the ESP - satisfaction of the GS can be measured by means of the Mutual False Nearest Neighbors index, which makes effective to practitioners theoretical derivations

Input-to-State Representation in linear reservoirs dynamics

Reservoir computing is a popular approach to design recurrent neural networks, due to its training simplicity and approximation performance. The recurrent part of these networks is not trained (e.g., via gradient descent), making them appealing for analytical studies by a large community of researchers with backgrounds spanning from dynamical systems to neuroscience. However, even in the simple linear case, the working principle of these networks is not fully understood and their design is usually driven by heuristics. A novel analysis of the dynamics of such networks is proposed, which allows the investigator to express the state evolution using the controllability matrix. Such a matrix encodes salient characteristics of the network dynamics; in particular, its rank represents an input-indepedent measure of the memory capacity of the network. Using the proposed approach, it is possible to compare different reservoir architectures and explain why a cyclic topology achieves favourable results as verified by practitioners

L'importanza dei tempi di osservazione nella formulazione dell'ipotesi ergodica

Questo elaborato tratta dell'ipotesi ergodica, problema centrale nell'ambito della giustificazione dei risultati della meccanica statistica, e dell'importanza che svolge in essa il tempo di osservazione. Dopo aver presentato varie formulazioni del problema ergodico, si esamina la questione dei tempi di ritorno e si mostra come il teorema di ricorrenza di Poincaré non sia in contraddizione con la possibilità del raggiungimento dell'equilibrio. Infine, l'analisi dell'apparente paradosso di Fermi-Pasta-Ulam e la discussione di alcune proposte di soluzione mostrano un'applicazione della trattazione astratta condotta precedentemente

Learning dynamical systems using dynamical systems: the reservoir computing approach

Dynamical systems have been used to describe a vast range of phenomena, including physical sciences, biology, neurosciences, and economics just to name a few. The development of a mathematical theory for dynamical systems allowed researchers to create precise models of many phenomena, predicting their behaviors with great accuracy. For many challenges of dynamical systems, highly accurate models are notably hard to produce due to the enormous number of variables involved and the complexity of their interactions. Yet, in recent years the availability of large datasets has driven researchers to approach these complex systems with machine learning techniques. These techniques are valuable in settings where no model can be formulated explicitly, but not rarely the working principles of these models are obscure and their optimization is driven by heuristics. In this context, this work aims at advancing the field by “opening the black-box” of data-driven models developed for dynamical systems. We focus on Recurrent Neural Networks (RNNs), one of the most promising and yet less understood approaches. In particular, we concentrate on a specific neural architecture that goes under the name of Reservoir Computing (RC). We address three problems: (1) how the learning procedure of these models can be understood and improved, (2) how these systems encode a representation of the inputs they receive, and (3) how the dynamics of these systems affect their performance. We make use of various tools taken from the theory of dynamical systems to explain how we can better understand the working principles of RC in dynamical systems, aiming at developing new guiding principles to improve their design

Unbiased choice of global clustering parameters for single-molecule localization microscopy

Abstract Single-molecule localization microscopy resolves objects below the diffraction limit of light via sparse, stochastic detection of target molecules. Single molecules appear as clustered detection events after image reconstruction. However, identification of clusters of localizations is often complicated by the spatial proximity of target molecules and by background noise. Clustering results of existing algorithms often depend on user-generated training data or user-selected parameters, which can lead to unintentional clustering errors. Here we suggest an unbiased algorithm (FINDER) based on adaptive global parameter selection and demonstrate that the algorithm is robust to noise inclusion and target molecule density. We benchmarked FINDER against the most common density based clustering algorithms in test scenarios based on experimental datasets. We show that FINDER can keep the number of false positive inclusions low while also maintaining a low number of false negative detections in densely populated regions

SICE national survey: current status on the adoption of laparoscopic approach to the treatment of colorectal disease in italy

The real di usion of laparoscopy for the treatment of colorectal diseases in Italy is largely unknown. The main purpose of the present study is to investigate among surgeons dedicated to minimally invasive surgery, the volume of laparoscopic colorectal procedures, the type of operation performed in comparison to traditional approach, the indication for surgery (benign and malignant) and to evaluate the di erent types of technologies used. A structured questionnaire was developed in collaboration with an international market research institute and the survey was published online; invitation to participate to the survey was issued among the members of the Italian Society of Endoscopic Surgery (SICE). 211 surgeons working in 57 surgical departments in Italy ful lled and answered the online survey. A total of 6357 colorectal procedures were recorded during the year 2015 of which 4104 (64.1%) were performed using a minimally invasive approach. Colon and rectal cancer were the most common indications for laparoscopic approach (83.1%). Left colectomy was the operation most commonly performed (41.8%), while rectal resection accounted for 23.5% of the cases. Overall conversion rate was 5.9% (242/4104). Full HD standard technology was available and routinely used in all the responders’ centers. The proportion of colorectal resec- tions that are carried out laparoscopically in dedicated centers has now reached valuable levels with a low conversion ra